Skip to content

Implement changelog-driven release workflow - #6822

Merged
masenf merged 8 commits into
mainfrom
claude/release-process-changelog-sot-tqmdpd
Jul 31, 2026
Merged

Implement changelog-driven release workflow#6822
masenf merged 8 commits into
mainfrom
claude/release-process-changelog-sot-tqmdpd

Conversation

@masenf

@masenf masenf commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Type of change

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Description

This PR implements a comprehensive changelog-driven release system that makes CHANGELOG.md the source of truth for publishing packages. The system replaces the previous tag-based release workflow with a more robust approach that handles failed publishes gracefully and supports both final and prerelease versions.

Key Changes

New Release Script (scripts/release.py)

  • Unified Python script replacing scattered bash helpers for release orchestration
  • Implements five subcommands:
    • detect: Identifies packages with untagged changelog versions
    • plan: Computes next versions based on release action (new-prerelease-, continued-prerelease, release-, etc.)
    • materialize: Writes planned versions to changelogs using towncrier
    • prepare-publish: Validates (package, version) pairs against changelog, branch rules, and tags
    • extract-notes: Exports changelog sections for GitHub release notes
  • Supports version computation for all release modes (alpha, patch, minor, major, post-release)
  • Implements collapse_prereleases() to merge accumulated alpha sections into final releases

New Workflows

  • release_from_changelog.yml: Publishes any changelog version lacking a git tag on pushes to main/r/** branches
  • Updated dispatch_release.yml: Materializes news fragments into changelogs at next version
    • Prerelease actions (new-prerelease-, continued-prerelease) push alphas directly to r/pre- branches
    • Release actions (release-*) open PRs for review before publishing
    • release-from-prerelease collapses alpha sections into final version
  • Updated publish.yml: Accepts package/version inputs, creates tags only after successful PyPI upload
  • Updated auto_release_internal.yml: Uses new publish workflow for internal packages

Supporting Scripts

  • open_release_pr.sh: Opens PR with changelog changes for release actions
  • push_prerelease.sh: Pushes prerelease versions to r/pre-* branches
  • create_release.sh: Creates GitHub releases with changelog notes
  • push_tag.sh: Pushes git tags after successful publish
  • verify_dist_version.sh: Validates built artifacts carry correct version

Documentation

  • Updated CONTRIBUTING.md with changelog-driven release process for maintainers

Design Rationale

  • Changelog as source of truth: Eliminates tag/changelog sync issues; failed publishes leave no tag behind
  • Graceful retry: Fix problems on top of changelog bump; next push retries without cleanup
  • Branch-aware publishing: Final versions only from main/r/hotfix/**; alphas from any branch
  • Prerelease trains: Accumulated alpha sections collapse into single final section (alpha headings never appear in published changelogs)
  • PR-based approval: Release actions open PRs; merging is the approval gate

Testing

  • Added comprehensive unit tests in tests/units/test_release.py covering:
    • Version parsing and extraction from changelogs
    • Next version computation for all release modes
    • Prerelease section collapsing
    • Git tag detection and version discovery
    • All five subcommands (detect, plan, materialize, prepare-publish, extract-notes)
    • Branch rule enforcement
    • Package discovery and category ordering

All tests pass with adequate coverage.

Checklist

  • Tests added for new functionality
  • uv run ruff check . and uv run ruff format . pass
  • uv run pyright reflex tests passes
  • Documentation updated (CONTRIBUTING.md)

https://claude.ai/code/session_01RytVevvLUNztmc2Shr9fgq

A package is now published exactly when the newest version heading in its
CHANGELOG.md has no corresponding git tag, and tags are only pushed after a
successful publish — a failed build/publish is retried by pushing a fix on
top of the changelog bump, with no tag or release cleanup.

- scripts/release.py: shared helper (detect / plan / materialize /
  prepare-publish / extract-notes) with changelog parsing, version planning,
  towncrier materialization, and alpha-section collapsing.
- release_from_changelog.yml (new): on push to main and r/**, publish any
  changelog version without a tag. Finals only publish from main or
  r/hotfix/**.
- publish.yml: now takes (package, version) via workflow_call/dispatch;
  validates against the changelog and branch rules, tags the local checkout,
  builds, publishes, then pushes the tag and creates the GitHub release with
  notes extracted from the changelog. Same filename and pypi environment, so
  PyPI trusted publishing is unchanged.
- dispatch_release.yml: same action modes, but now materializes changelogs.
  Prerelease actions push alphas straight to an r/pre-<date> branch (no
  approval); release-* actions open a PR whose review+merge is the release
  approval. release-from-prerelease collapses alpha sections into the single
  final version section, so alphas never appear in a final changelog. The
  dispatch-release-approval environment gate is retired.
- auto_release_internal.yml: internal packages publish through publish.yml
  with an auto patch-bump version computed inside the per-package publish
  lock; no more tag-before-publish.
- Obsolete per-workflow scripts removed; CONTRIBUTING.md documents the flow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RytVevvLUNztmc2Shr9fgq
@masenf
masenf requested a review from a team as a code owner July 30, 2026 00:51
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

Comment thread .github/workflows/publish.yml
@greptile-apps

greptile-apps Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR replaces tag-driven releases with a changelog-driven release pipeline.

  • Adds unified release planning, materialization, validation, note extraction, and artifact verification in scripts/release.py.
  • Adds workflows for changelog detection, gated package publishing, post-upload tagging, and GitHub release creation.
  • Updates dispatch and internal-package release workflows, supporting scripts, tests, and maintainer documentation.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the previously reported subpackage artifact-path issue was invalidated because the pinned uv workspace build writes those artifacts to the repository-root dist directory used by verification and publishing.

Important Files Changed

Filename Overview
scripts/release.py Centralizes changelog parsing, version planning, materialization, publish validation, release-note extraction, dependency pinning, and artifact verification.
.github/workflows/publish.yml Reworks publishing into separate build, approval-gated upload, and post-upload tag-and-release stages.
.github/workflows/release_from_changelog.yml Detects untagged changelog versions and coordinates package publishing while preserving the reflex/reflex-base ordering constraint.
.github/workflows/dispatch_release.yml Replaces immediate tag-based releases with changelog materialization followed by prerelease branch pushes or reviewed release pull requests.
tests/units/test_release.py Adds unit coverage for release planning, changelog handling, branch policy, package detection, and publish preparation.

Reviews (9): Last reviewed commit: "Stamp changelog dates in the release tim..." | Re-trigger Greptile

@codspeed-hq

codspeed-hq Bot commented Jul 30, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 26 untouched benchmarks
⏩ 8 skipped benchmarks1


Comparing claude/release-process-changelog-sot-tqmdpd (cdee059) with main (fa3e67a)

Open in CodSpeed

Footnotes

  1. 8 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

claude added 3 commits July 30, 2026 18:37
… fail-closed guards

Restructure publish.yml into three stages so every PyPI upload requires
human approval while builds stay automatic:
- build (unprivileged): validate against changelog/branch rules/lockstep,
  tag locally, build, verify artifact core-metadata versions, extract
  release notes, upload everything as a workflow artifact.
- publish: gated by the pypi environment's required reviewers — alphas and
  internal packages included. Holds the only id-token privilege, runs no
  repository scripts and resolves no dependencies (only uv publish on the
  pre-built artifact), and fails closed if it cannot prove the pypi
  environment actually requires reviewers.
- tag-and-release: pushes the tag and creates the GitHub release only after
  a successful upload.

Other hardening:
- Branch policy: prereleases may only publish from r/pre-* or r/hotfix/**
  (no longer any r/** or main); push triggers narrowed to match.
- reflex/reflex-base lockstep enforced fail-closed at detect and publish
  time, and reflex publishes in a second phase only after the rest of the
  batch succeeds; a report job turns partial releases into one loud failure.
- Every publish entry point now shares the publish-<package> concurrency
  group, closing the dispatch-vs-automatic double-publish race.
- changelog CI rejects hand-added version headings (they are the publish
  trigger); release/* branches and the skip-changelog label are exempt.
- GitHub release "Latest" is only set when the version is actually the
  newest final (hotfixes of old lines no longer steal the badge).
- scripts/release.py grows pin-reflex-base (replaces the sed script,
  asserts exactly one rewrite) and verify-dist (compares each artifact's
  METADATA/PKG-INFO Version, replacing the filename heuristic); dates are
  UTC; new-prerelease-* refuses to silently abandon an in-progress alpha
  train; non-alpha prereleases get a clear error.
- Script dependencies are locked with hashes (scripts/release.py.lock,
  used automatically by uv run --script).
- Git pushes authenticate via gh's credential helper instead of a token in
  the remote URL, and materialization commits stage only changelogs and
  news fragments with an explicit empty-diff guard.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RytVevvLUNztmc2Shr9fgq
The pypi-gate job's permissions block implicitly set contents: none, which
could 403 the environments API read and permanently block publishing with a
misleading error. Grant contents: read and give the two failure modes
distinct messages (token/configuration problem vs missing reviewers).

Also update the comments and docs that still described the pre-approval
iteration: the dispatch workflow header and CONTRIBUTING no longer claim
alphas publish without approval or that PR review alone approves a release —
every upload waits for pypi environment approval; PR merge is how final
versions land.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RytVevvLUNztmc2Shr9fgq
…ecksums, strict batch report

- release.py's header now pins exact versions matching release.py.lock, and
  every workflow call site runs uv with --no-config --locked so any
  header/lock drift fails loudly instead of silently re-resolving
  (--no-config keeps the moving [tool.uv] exclude-newer window in the repo's
  pyproject from invalidating the script lock daily).
- The changelog heading guard no longer re-implements the parser in grep:
  a new check-headings subcommand diffs version headings against the PR base
  using the exact parser the release pipeline publishes from, closing the
  whitespace/case bypasses. Its escape hatch is a dedicated
  changelog-version-edit label — skip-changelog only waives the fragment
  check and no longer disables the publish-trigger guard.
- The build job writes a SHA256SUMS manifest covering dist/* and the release
  notes; the gated publish job verifies it with coreutils after the approval
  pause, so what the human approved is byte-identical to what uploads.
- The release batch report treats any leg result other than success/skipped
  (cancelled, timed_out, rejected approval) as a failure instead of
  reporting green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RytVevvLUNztmc2Shr9fgq

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 8 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread scripts/release.py

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed against the latest diff

Tip: instead of fixing issues one by one fix them all with cubic
Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Re-trigger cubic

Comment thread .github/scripts/publish/create_release.sh
Comment thread .github/scripts/dispatch_release/open_release_pr.sh
Comment thread .github/workflows/publish.yml
Comment thread .github/scripts/dispatch_release/push_prerelease.sh
Comment thread .github/workflows/dispatch_release.yml
Comment thread .github/workflows/dispatch_release.yml
Comment thread CONTRIBUTING.md
Comment thread .github/scripts/dispatch_release/push_prerelease.sh
claude added 2 commits July 30, 2026 21:35
Dispatching with no packages checked no longer errors: the plan step
auto-detects what is releasable. For most actions that means packages with
unmaterialized news fragments (fragments in the repo-root news/ select
reflex-base, which releases reflex alongside it). release-from-prerelease
instead selects packages whose changelog is topped by an alpha, since the
train's fragments were already consumed into the alpha sections. The plan
summary records whether the selection was explicit or auto-detected, and
planning still fails when nothing is selectable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RytVevvLUNztmc2Shr9fgq
git pathspecs containing a wildcard must match complete file paths, so
'packages/*/news' never matches anything (directory-recursive matching only
applies to literal directory pathspecs) and git add hard-fails the Push
prerelease branch step. Stage the changelog files explicitly (globs on file
paths work, and cover first-release changelog creations) and let
`git commit -a` record towncrier's fragment deletions — already staged via
its git rm — plus any tracked change left unstaged, while still never
committing untracked stray files. The no-change guard now checks both the
index and the working tree.

Verified end-to-end against a local bare origin: materialize-all followed by
push_prerelease.sh commits exactly the changelog rewrites and fragment
deletions and pushes the r/pre-<date> branch; the empty-diff and
continued-prerelease branch guards still fail cleanly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RytVevvLUNztmc2Shr9fgq

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic

Comment thread .github/scripts/dispatch_release/open_release_pr.sh
Comment thread .github/scripts/dispatch_release/push_prerelease.sh
claude added 2 commits July 30, 2026 23:07
uv build writes a .gitignore into dist/, and pathlib's glob("*") includes
dotfiles while the shell glob in `uv publish dist/*` (and `sha256sum
dist/*`) does not — so verify-dist failed the build with "unexpected
artifact type: .gitignore" on the first real alpha train. Verify exactly the
set of files publish will upload: non-hidden regular files only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RytVevvLUNztmc2Shr9fgq
CI runners run on UTC, so an evening dispatch dated the materialized
changelog headings with tomorrow's date. Dates now come from a single
RELEASE_TIMEZONE (America/Los_Angeles): release.py computes the date once
per materialize run and passes it to towncrier explicitly (--date) and to
the alpha collapse, and the r/pre-<date> branch name uses the same timezone
so branch names and headings can never disagree. tzdata is declared for
Windows so zoneinfo resolves everywhere.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RytVevvLUNztmc2Shr9fgq

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic

Comment thread .github/scripts/dispatch_release/push_prerelease.sh
Comment thread scripts/release.py
@masenf
masenf merged commit 0c7bdfb into main Jul 31, 2026
107 of 108 checks passed
@masenf
masenf deleted the claude/release-process-changelog-sot-tqmdpd branch July 31, 2026 00:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants